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Security,  Committee  on  Appropriations, 
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ANTHRAX  DETECTION 

DHS  Cannot  Ensure  That  Sampling 
Activities  Will  Be  Validated 


Why  GAO  Did  This  Study 

In  September  and  October  2001, 
contaminated  letters  laced  with 
Bacillus  anthracis  were  sent 
through  the  mail  to  two  U.S. 
senators  and  members  of  the 
media.  Postal  facilities  in  New 
Jersey,  Washington,  D.C.,  and 
elsewhere  became  heavily 
contaminated.  The  anthrax 
incidents  highlighted  major  gaps  in 
civilian  preparedness  to  detect 
anthrax  contamination  in  buildings. 
GAO  was  asked  to  describe  and 
assess  federal  agencies’  activities 
to  detect  anthrax  in  postal 
facilities,  assess  the  results  of 
agencies’  testing,  and  assess 
whether  agencies’  detection 
activities  were  validated. 


What  GAO  Recommends 


GAO  is  not  making  any  new 
recommendations. 


www.gao.gov/cgi-bin/getrpt7GAO-07-687T. 

To  view  the  full  product,  including  the  scope 
and  methodology,  click  on  the  link  above. 
For  more  information,  contact  Keith  Rhodes 
at  (202)  512-6412  or  rhodesk@gao.gov. 


What  GAO  Found 

Federal  agencies  conducted  several  sampling  activities,  including  developing 
a  sampling  strategy  and  collecting,  transporting,  extracting,  and  analyzing 
samples.  They  primarily  collected  samples  from  specific  areas,  such  as  mail 
processing  areas,  using  their  judgment  about  where  anthrax  would  most 
likely  be  found — that  is,  targeted  sampling.  The  agencies  did  not  use 
probability  sampling,  which  would  have  allowed  agencies  to  determine,  with 
some  defined  level  of  confidence,  when  all  results  are  negative,  whether  a 
building  is  contaminated. 

The  results  of  the  agencies’  testing  in  286  postal  facilities  were  largely 
negative — no  anthrax  was  detected.  However,  agencies  did  not  use  validated 
sample  collection  and  analytical  methods.  Thus,  there  can  be  little 
confidence  in  negative  results.  With  a  validated  process,  agencies  and  the 
public  could  be  reasonably  confident  that  any  test  results  generated  by  that 
process  would  be  reliable. 

The  Department  of  Homeland  Security  (DHS)  is  the  principal  agency 
responsible  for  coordinating  the  federal  response.  Thus,  in  its  2005  report, 

GAO  recommended  that  the  Secretary  of  Homeland  Security  develop  a 
coordinated  approach  to  improve  the  overall  process  for  detecting  anthrax 
and  increase  confidence  in  negative  test  results  generated  by  that  process. 

DHS  stated  that  while  it  has  overall  responsibility  for  coordinating  the 
federal  response  during  future  biological  attacks,  other  agencies  have  the 
lead  responsibility  for  validation.  Therefore,  uncertainty  over  which  agency 
would  take  the  lead  role — that  is,  who  is  in  charge — in  improving  the  overall 
process  for  detecting  anthrax,  including  validation  of  the  methods, 
continued  after  GAO  issued  its  report. 

On  the  basis  of  these  uncertainties,  GAO  recommended  in  its  May  9,  2006, 
testimony  that  DHS’s  approach  to  validating  the  overall  process  start  with  a 
strategic  plan  that  would  include  a  road  map  outlining  how  individual 
agencies’  efforts  would  lead  to  the  validation  of  the  individual  activities  as 
well  as  the  overall  process,  noting  that  such  a  plan  would  assist  DHS  in 
monitoring  progress  and  measuring  agency  performance  toward  improving 
the  detection  of  anthrax  and  other  prioritized  threat  agents. 

While  DHS  generally  agreed  with  these  recommendations,  it  stated  that  it 
cannot  ensure  validation  studies  would  be  done,  since  “there  are  legal 
limitations  in  DHS  authority  to  direct  the  activities  of  other  agencies.”  Also, 
since  validation  would  require  a  sustained  effort  over  a  long  period,  DHS 
noted  that  it  could  not  mandate  commitment  of  other  agencies’  funds,  over 
which  it  has  no  control. 

Until  responsibility  is  accepted  for  ensuring  that  sampling  activities  will  be 

validated,  the  fate  of  the  validation  process  will  remain  uncertain.  Without 

validation,  if  another  anthrax  attack  were  to  occur  tomorrow,  federal  civilian 

agencies  would  not  be  able  to  conclude  with  any  given  level  of  statistical 

confidence,  in  cases  of  negative  results,  that  a  building  is  free  of 

contamination.  „  .  .  .  .....  n... 

_ United  States  Government  Accountability  Office 


Mr.  Chairman  and  Members  of  the  Subcommittee: 


We  are  pleased  to  be  here  today  to  discuss  our  findings  on  anthrax 
detection  testing.  The  threat  of  bioterrorism  has  been  recognized  for  a 
considerable  time.  Long  before  the  anthrax  attacks  of  2001,  several  hoax 
letters  indicating  the  presence  of  anthrax  had  been  mailed  to  federal  and 
state  agencies,  as  well  as  to  private  sector  organizations.  These  events 
raised  the  possibility  that  facilities  could  become  contaminated  and  would 
therefore  have  to  be  evaluated  for  environmental  contamination.  However, 
federal  agencies  were  not  fully  prepared  to  deal  with  environmental 
contamination — that  is,  anthrax  released  through  the  mail — including  the 
potential  for  multiple  dispersals  in  indoor  environments. 

In  September  and  October  2001,  contaminated  letters  laced  with  Bacillus 
anthracis  were  sent  through  the  mail  to  two  U.S.  senators,  Thomas 
Daschle  and  Patrick  Leahy,  and  members  of  the  media.1  The  postal 
facilities  in  New  Jersey  and  Washington,  D.C.,  that  processed  the  senators’ 
letters  became  heavily  contaminated.  Other  mail  routed  through  these 
facilities,  as  well  as  additional  facilities  in  the  postal  network,  also  became 
contaminated.  In  addition,  numerous  federal  facilities  in  the  Washington, 
D.C.,  area  were  later  found  to  be  contaminated.  The  letters  led  to  the  first 
cases  of  anthrax  disease  related  to  bioterrorism  in  the  United  States.  In  all, 
22  individuals  contracted  anthrax  disease  in  four  states — Connecticut, 
Florida,  New  Jersey,  and  New  York — as  well  as  in  Washington,  D.C.  Five 
of  these  22  individuals  died. 

The  anthrax  incidents  in  September  and  October  2001  highlighted  major 
gaps  in  civilian  preparedness  to  detect  and  respond.  In  today’s  testimony, 

I  will  discuss  our  findings  concerning  anthrax  sampling  activities, 
recommendations  we  made,  and  a  major  issue  we  identified — the 
Department  of  Homeland  Security  (DHS)  cannot  ensure  and  guarantee 
that  sampling  activities  will  be  validated. 


1 Bacillus  anthracis  is  the  microorganism  that  causes  the  disease  known  as  anthrax. 
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In  developing  this  testimony,  we  relied  on  our  prior  work.2  We  conducted 
our  review  in  accordance  with  generally  accepted  government  auditing 
standards. 

Results  in  Brief 

Federal  agencies  responsible  for  responding  to  the  2001  anthrax  incidents 
adopted  a  targeted  sampling  strategy  that  they  based  on  their  best 
judgment  at  the  time.  They  primarily  collected  samples  from  specific 
areas,  such  as  mail-processing  areas,  using  their  judgment  about  where 
anthrax  would  most  likely  be  found.  Such  judgments  can  be  effective  in 
some  situations — for  example,  in  determining  whether  a  facility  is 
contaminated  when  information  on  the  source  of  potential  contamination 
is  definitive.  However,  in  the  case  of  a  negative  finding,  when  the  source  of 
potential  contamination  is  not  definitive,  the  basic  question — Is  this 
building  contaminated? — will  remain  unanswered.  Therefore,  in  the  case 
of  a  negative  result,  a  different  strategy,  probability  sampling,  is  needed. 
Probability  sampling  would  have  allowed  agencies  to  determine  whether 
the  building  was  contaminated  with  some  defined  level  of  confidence. 

The  federal  agencies — CDC,  EPA,  and  USPS — involved  in  sampling  the 
postal  facilities  in  2001  to  detect  anthrax  undertook  several  activities. 

These  included  development  of  a  sampling  strategy  followed  by  collection 
of  samples  using  a  variety  of  methods,  transporting  and  extracting,  and 
analysis  of  the  samples.  Neither  these  activities  nor  the  overall  process 
was  validated  for  anthrax  testing.  Consequently,  the  agencies  were 
challenged  by  the  limited  information  available  for  reliably  choosing  one 
method  over  another  and  the  lack  of  information  on  the  detection  limit  to 
use  when  evaluating  negative  results. 

The  results  of  the  CDC,  EPA,  and  USPS  testing  in  286  postal  facilities  were 
largely  negative.3  Of  the  286  facilities,  23  tested  positive.  For  2  of  these 

23  facilities,  test  results  were  negative  at  first  but  positive  on  a  subsequent 

2GAO,  Anthrax  Detection:  Agencies  Need  to  Validate  Sampling  Activities  in  Order  to 
Increase  Confidence  in  Negative  Results,  GAO-05-251  (Washington  D.C.:  Mar  27,  2005); 
Anthrax  Detection:  Agencies  Need  to  Validate  Sampling  Activities  in  Order  to  Increase 
Confidence  in  Negative  Results,  GAO-05-493T  (Washington  D.C.:  Apr.  5,  2005);  and 
Anthrax:  Federal  Agencies  Have  Taken  Some  Steps  to  Validate  Sampling  Methods  and  to 
Develop  a  Next-Generation  Anthrax  Vaccine,  GAO-06-756T  (Washington  D.C.:  May  9, 

2006). 

3While  the  Federal  Bureau  of  Investigation  (FBI)  also  collected  samples,  we  did  not  include 
the  results  of  its  testing  due  to  its  ongoing  criminal  investigation. 
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testing.  However,  in  1  of  these  facilities — the  Wallingford,  Connecticut, 
facility — it  was  not  until  the  fourth  testing  that  positive  results  were 
obtained. 

The  federal  agencies’  activities  to  detect  anthrax  contamination  were  not 
validated.  Without  validation,  the  sampling  activities  could  have  been 
based  on  false  assumptions. 

For  example,  the  lack  of  validated  sample  collection  methods  means  that 
it  is  not  known  how  many  spores  a  particular  method  will  collect  from  a 
surface  and,  thus,  which  method  is  appropriate  for  a  given  situation.  Using 
an  ineffective  method  or  procedure  could  result  in  a  finding  of  no 
contamination  when  in  fact  there  is  contamination — a  false  negative. 

Validating  the  overall  process,  as  well  as  the  individual  activities,  is 
important  because  operational  and  health-related  decisions  are  made  on 
the  basis  of  testing  results  generated  by  that  process.  In  addition, 
validation  would  offer  assurance  that  the  results  of  using  a  particular 
method,  which  is  part  of  that  process,  are  robust  enough  to  be  reproduced, 
regardless  of  which  agency,  contractor,  or  laboratory  is  involved.  Thus, 
agencies  and  the  public  could  be  reasonably  confident  that  any  test  results 
generated  by  a  process  that  includes  that  method  would  be  reliable  and,  in 
particular,  that  any  negative  results  would  mean  that  a  sample  was  free 
from  contamination  (within  the  method’s  limits  of  detection). 

Given  the  lack  of  validated  methods  for  detecting  anthrax  contamination 
in  facilities,  we  recommended  that  the  Secretary  of  Homeland  Security 
develop  a  coordinated  approach  to  (1)  improve  the  overall  process  for 
detecting  anthrax  and  (2)  increase  confidence  in  negative  test  results 
generated  by  that  process.  This  approach  would  include  working  with 
agencies  to  ensure  that  appropriate  validation  studies  of  the  overall 
process  of  sampling  activities,  including  the  methods,  are  conducted. 
Specifically,  we  recommended  that  the  Secretary 

1.  take  a  lead  role  in  promoting  and  coordinating  the  activities  of  the 
various  agencies  that  have  the  technical  expertise  related  to 
environmental  testing; 

2.  ensure  that  a  definition  of  validation  is  developed  and  agreed  on; 

3.  guarantee  that  the  overall  process  of  sampling  activities,  including 
methods,  is  validated  so  that  performance  characteristics,  including 
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limitations,  are  clearly  understood  and  results  can  be  correctly 
interpreted; 

4.  see  that  appropriate  investments  are  made  in  empirical  studies  to 
develop  probability-based  sampling  strategies  that  take  into  account 
the  complexities  of  indoor  environments; 

5.  ensure  that  appropriate,  prioritized  investments  are  made  for  all 
biothreat  agents;  and 

6.  make  sure  that  agency  policies,  procedures,  and  guidelines  reflect  the 
results  of  such  efforts.4 

When  we  issued  our  report,  CDC,  DHS,  and  USPS  agreed  with  our 
conclusion — dial  methods  for  detecting  anthrax  contamination  in  facilities 
were  not  validated — and  with  the  thrust  of  our  recommendations — calling 
for  a  coordinated,  systematic  effort  to  validate  the  methods  to  be  used  for 
such  testing.  But  they  (1)  disagreed  with  or  expressed  concern  about  our 
conclusions  or  the  recommendation  dealing  with  targeted  versus 
probability  sampling,  (2)  emphasized  that  validated  testing  methods  for 
anthrax  were  not  available  in  2001  and  that  federal  and  state  organizations 
did  the  best  they  could  under  the  circumstances,  and  (3)  identified  factors 
or  issues  that  need  to  be  considered  in  validating  testing  methods. 

In  addition,  uncertainty  over  which  agency  would  take  the  lead  role — that 
is,  who  is  in  charge — in  improving  the  overall  process  for  detecting 
anthrax,  and  how  studies  were  to  be  funded,  continued  after  we  issued  our 
report.  DHS  stated  that  while  it  has  overall  responsibility  for  coordinating 
the  federal  response  during  future  biological  attacks,  EPA  had  the 
“primary  responsibility  for  establishing  the  strategies,  guidelines,  and 
plans  for  the  recovery  from  a  biological  attack,”  while  the  Department  of 
Health  and  Human  Services  (HHS)  had  the  lead  role  for  any  related  public 
health  response  and  guidelines.  DHS  also  stated  that  it  coordinated 
regularly  with  EPA’s  National  Homeland  Research  Center  to  exchange 
information  on  research  needs  and  to  discuss  priorities  and  gaps  for  a 
wide  range  of  security-related  research  areas.  DHS  stated  that  it  would 
coordinate  with  EPA  to  ensure  that  appropriate  investments  were  made  to 
explore  improved  sampling.  However,  it  is  unclear  to  us  how  DHS  would 
ensure  that  appropriate  prioritized  investments  are  made  for  all  biothreat 
agents  and  how  such  priorities  and  gaps  would  be  addressed. 


4GAO-05-251,  pp.  82-83. 
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On  the  basis  of  these  uncertainties,  we  recommended  in  our  May  9,  2006, 
testimony  that  DHS’s  approach  to  validating  the  overall  process  start  with 
a  strategic  plan  that  includes  a  road  map  outlining  how  individual 
agencies’  efforts  would  lead  to  the  validation  of  the  individual  activities  as 
well  as  the  overall  process,  noting  that  such  a  plan  would  assist  DHS  in 
monitoring  progress  and  measuring  agency  performance  toward 
improving  the  detection  of  anthrax  and  other  prioritized  threat  agents.5  6 

On  May  19,  2006,  DHS  officials  stated  that  DHS  cannot  ensure  and 
guarantee  that  validation  studies  would  be  done,  since  this  is  a  shared 
responsibility  among  different  agencies.  DHS  stated  that  “there  are  legal 
limitations  in  DHS  authority  to  direct  the  activities  of  other  agencies.” 

Also,  since  validation  would  require  a  sustained  effort  over  a  long  period, 
these  officials  noted  that  they  could  not  mandate  commitment  of  other 
agencies’  funds,  because  of  legal  and  budgetary  limitations. 

DHS  officials  told  us  in  July  2006  that  they  recognize  that  DHS  is  the 
principal  agency  responsible  for  coordinating  the  federal  response  and 
they  would  work  with  a  good  faith  effort  toward  developing  a  strategy  for 
validation  studies  and  a  road  map  by  the  end  of  calendar  year  2006, 
outlining  how  individual  agencies’  efforts  would  lead  to  the  validation  of 
the  overall  sampling  process.  On  March  27,  2007,  DHS  told  us  that  it  had 
developed  a  working  draft  of  the  strategic  plan  and  the  road  map  by 
December  2006  but  it  could  not  share  these  with  us  because  they  were  not 
final.6 

Until  responsibility  is  accepted  for  ensuring  that  sampling  activities  will  be 
validated,  the  fate  of  the  validation  process  will  remain  uncertain.  Without 
validation,  if  another  anthrax  attack  were  to  occur  tomorrow,  federal 
civilian  agencies  would  not  be  able  to  conclude  with  any  given  level  of 
statistical  confidence,  in  cases  of  negative  results,  that  a  building  is  free  of 
contamination. 


Background 


In  October  2001,  an  American  Media  Incorporated  employee  died  from 
inhalation  anthrax  disease.  In  the  same  month,  contaminated  letters  laced 


5GAO-06-756T 

6A1so  on  March  27,  2007,  DHS  officials  gave  us  a  short  status  report  on  the  Anthrax 
Sampling  Working  Group.  However,  we  could  not  evaluate  the  significance  of  the  activities 
it  summarized  without  the  strategic  plan. 
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with  Bacillus  anthracis,  or  anthrax  spores,  were  sent  through  the  mail  to 
Senators  Thomas  Daschle  and  Patrick  Leahy.  The  response  to  the  incident 
in  the  American  Media  Incorporated  building  in  Florida  in  September  2001 
led  to  the  identification  of  mail  as  the  potential  source  of  contamination; 
eventually,  it  led  to  the  sampling  of  the  postal  facilities.  The  agencies 
began  sampling  on  October  12,  2001,  in  Florida  and  stopped  on  April  21, 
2002,  when  the  Wallingford,  Connecticut,  facility  was  sampled  for  the  last 
time.  The  letters  led  to  the  first  cases  of  anthrax  disease  related  to 
bioterrorism  in  the  United  States.  In  all,  22  individuals  contracted  anthrax 
disease  in  Connecticut,  Florida,  New  Jersey,  and  New  York,  as  well  as  in 
Washington,  D.C.,  and  5  died. 

The  federal  agencies  involved  in  the  response  in  the  postal  facilities  have 
different  responsibilities.  CDC  and  state  and  local  health  departments 
primarily  provided  public  health  advice  and  assistance  to  USPS.  CDC  has 
primary  responsibility  for  national  surveillance  of  specific  diseases, 
including  anthrax;  it  also  conducts  epidemiologic  investigations  to 
determine,  among  other  things,  the  source  of  the  disease,  and  it 
participates  in  environmental  sample  collection  and  analysis  activities. 

The  FBI  is  responsible  for  criminal  investigations  involving  interstate 
commerce  and  the  mail  and  crimes  committed  on  federal  property.  EPA  is 
the  nation’s  lead  agency  for  responding  to  a  release  of  hazardous 
substances  into  the  environment  and  subsequent  decontamination. 

On  October  8,  2001,  the  President  created  the  Office  of  Homeland  Security 
to  develop  and  coordinate  a  comprehensive  national  strategy  for  dealing 
with  domestic  terrorist  threats  or  attacks.  The  office,  which  had  limited 
involvement  in  the  2001  response,  was  superseded  by  the  Homeland 
Security  Act  of  2002,  which  transferred  many  of  its  functions  to  DHS, 
which  became  operational  in  2003.  DHS  was  created  by  combining  many 
previously  separate  agencies  and  is  assigned  a  lead  role  in  coordinating 
the  efforts  of  federal  agencies  that  respond  to  acts  of  terrorism  in  the 
United  States. 


Major  Findings 


Sampling  Strategy  The  federal  agencies  primarily  used  a  targeted  strategy — they  collected 

samples  from  specific  areas  considered  more  likely  to  be  contaminated, 
based  on  judgments.  Such  judgments  can  be  effective  in  some  situations — 
for  example,  in  determining  whether  a  facility  is  contaminated  when 
information  on  the  source  of  potential  contamination  is  definitive. 
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However,  in  the  case  of  a  negative  finding,  when  the  source  of  potential 
contamination  is  not  definitive,  the  basic  question — Is  this  building 
contaminated? — will  remain  unanswered. 

CDC  and  USPS  officials  said  that  they  used  a  targeted  strategy  for  several 
reasons,  including  limitations  on  how  many  samples  could  be  collected 
and  analyzed.  They  also  said  that  in  2001,  they  lacked  the  data  from 
empirical  research  to  develop  an  initial  sampling  strategy  that 
incorporated  probability  sampling.  We  disagree  with  this  interpretation. 
Probability  sampling  is  statistically  based  and  does  not  depend  solely  on 
empirical  criteria  regarding  the  details  of  possible  contamination. 

The  situation  in  2001  was  unique,  and  the  agencies  were  not  fully  prepared 
to  deal  with  environmental  contamination.  In  the  future,  if  the  agencies 
decide  to  use  a  targeted  rather  than  a  probability  sampling  strategy,  they 
must  recognize  that  they  could  lose  a  number  of  days  if  their  targeted 
sampling  produces  negative  test  results.  In  this  case,  additional  samples 
would  have  to  be  collected  and  analyzed,  resulting  in  the  loss  of  critical 
time  for  public  health  interventions.  This  was  so  at  the  Wallingford  postal 
facility  in  the  fall  of  2001,  when  about  3  weeks  elapsed  between  the  time 
the  first  sampling  took  place  and  the  results  of  the  fourth  testing,  which 
revealed  positive  results.  Furthermore,  about  5  months  elapsed  between 
the  time  of  the  first  sampling  event  and  the  time  anthrax  was  found  in  the 
Wallingford  facility’s  high-bay  area. 

Therefore,  strategies  that  include  probability  sampling  need  to  be 
developed  in  order  to  provide  statistical  confidence  in  negative  results. 
Further,  even  if  information  on  all  the  performance  characteristics  of 
methods  is  not  yet  available,  a  probability  sampling  strategy  could  be 
developed  from  assumptions  about  the  efficiency  of  some  of  the  methods. 
And  even  if  precise  data  are  not  available,  a  conservative,  approximate 
number  could  be  used  for  developing  a  sampling  strategy.  This  would  give 
agencies  and  the  public  greater  confidence  in  negative  test  results  than 
was  associated  with  the  sampling  strategy  used  in  2001. 


Sampling  Methods  CDC,  EPA,  and  USPS,  the  federal  agencies  involved  in  sampling  the  postal 

facilities  in  2001  to  detect  anthrax,  undertook  several  activities.  These 
included  development  of  a  sampling  strategy  followed  by  collecting 
samples,  using  a  variety  of  methods,  and  transporting,  extracting,  and 
analyzing  the  samples.  Neither  these  activities  nor  the  overall  process  was 
validated  for  anthrax  testing.  Consequently,  the  agencies  were  challenged 
by  limited  information  for  reliably  choosing  one  method  over  another  and 
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by  lack  of  information  on  the  detection  limit  to  use  when  evaluating 
negative  results. 

Federal  agencies  used  different  methods  for  collecting  samples.  While 
USPS  generally  used  dry  swabs  to  collect  samples  (the  least  effective 
method),  CDC  and  EPA  used  multiple  methods — dry  swabs,  premoistened 
swabs,  wet  wipes,  and  a  high-efficiency  particulate  air  (HEPA)  vacuum — 
in  various  combinations  or  alone. 

However,  none  of  the  agencies’  collection  methods  were  evaluated  for 
anthrax  detection  in  environmental  samples.  In  the  absence  of  empirical 
research,  agencies  had  no  information  available  for  reliably  choosing  one 
method  over  another  and  no  information  on  the  limits  of  detection  to  use 
when  evaluating  negative  results. 


Results  of  Testing  The  majority  of  the  samples  collected  from  the  postal  facilities  tested 

negative.  In  all,  federal  agencies  collected  about  10,000  samples  during 
initial  testing.  It  is  interesting  that  of  the  9,807  samples  that  the  agencies 
collected,  more  than  98  percent,  or  9,648,  were  negative;  a  little  more  than 
1  percent,  or  159,  were  positive.  In  all,  286  facilities  were  tested  for 
anthrax  contamination.  Of  these,  Brentwood,  Trenton,  and  Morgan  were 
primary  facilities;  that  is,  these  3  facilities  processed  the  original  letters 
containing  the  anthrax. 

The  results  of  the  CDC,  EPA,  and  USPS  testing  in  286  postal  facilities  were 
largely  negative.  Of  286  facilities,  23  tested  positive.  For  2  of  these 
23  facilities,  test  results  were  negative  at  first  but  positive  on  a  subsequent 
testing.  However,  in  1  of  these  facilities — the  Wallingford,  Connecticut, 
facility — it  was  not  until  the  fourth  testing  that  positive  results  were 
obtained. 

Testing  results  differed  between  the  primary  facilities  and  Wallingford.  In 
the  3  primary  facilities,  results  were  positive  each  time  a  facility  was 
tested,  with  the  important  exception  of  the  two  quick  tests  in  Brentwood. 
In  Wallingford,  considered  less  likely  to  be  contaminated,  results  were 
positive  only  on  the  fourth  sampling.  These  results  underscore  the 
importance  of  retesting  and  cast  doubt  on  the  efficiency  of  the  judgmental 
sampling  strategy. 

Of  the  263  facilities  that  tested  negative,  only  9  were  sampled  more  than 
once.  A  facility  in  West  Trenton  tested  negative,  even  though  an  employee 
had  contracted  cutaneous  anthrax.  The  facility  in  West  Trenton  was  tested 
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twice  by  the  FBI  and  once  by  CDC,  during  which  a  total  of  57  samples 
were  collected,  with  negative  results. 

Final,  or  confirmed,  results  will  be  negative  if  contamination  is  not  present 
in  a  facility.  However,  a  result  can  be  erroneously  negative  for  several 
other  reasons,  such  as  (1)  the  sampling  method  was  not  efficient  enough, 
(2)  samples  were  not  collected  from  places  where  contamination  was 
present,  (3)  not  enough  samples  were  collected,  (4)  not  enough  spores 
were  recovered  from  the  sample  material,  or  (5)  analysis  of  the  sample 
extract  was  not  sensitive  enough  to  detect  anthrax  spores  that  were 
present. 


Conclusions  The  agencies  that  sampled  postal  facilities  in  2001 — USPS,  CDC,  and 

EPA — did  not  use  validated  sample  collection  and  analysis  methods  to 
perform  their  tests.  According  to  these  agencies,  validated  methods  were 
not  available  at  that  time.  They  conducted  several  interdependent 
activities,  including  sample  strategy  development,  followed  by  sample 
collection,  transportation,  and  analysis  of  the  samples  to  detect  anthrax. 
Neither  these  activities  nor  the  overall  process  had  been  validated  for 
anthrax  testing. 

Validation  is  a  formal,  empirical  process  in  which  an  authority  determines 
and  certifies  the  performance  characteristics  of  a  given  method. 

Therefore,  investments  are  also  needed  to  validate  these  methods,  as  well 
as  the  overall  anthrax  detection  process.  Validating  the  overall  process,  as 
well  as  the  individual  activities,  is  important  because  operational  and 
health-related  decisions  are  made  on  the  basis  of  testing  results  that  the 
process  generates. 

CDC  and  USPS  officials  said  that  they  used  targeted  sampling;  that  is,  they 
collected  samples  from  specific  areas  considered — based  on  agencies’ 
technical  judgments — more  likely  to  be  contaminated.  Such  judgments  can 
be  effective  in  some  situations,  for  example,  in  determining  the  source  of 
contamination  in  a  disease  outbreak  investigation,  provided  results  are 
positive.  However,  if  the  results  are  negative,  the  basic  question — Is  this 
building  contaminated? — cannot  be  answered  with  statistical  confidence. 

When  the  level  of  contamination  is  extremely  high  and  dispersed  in  a 
facility,  the  method  of  sampling  (for  example,  wipes  versus  swabs)  is  not 
as  critical  if  the  purpose  is  to  find  some  contaminant.  However,  at  lower 
levels,  a  way  of  interpreting  the  significance  of  negative  results  is  needed, 
and  this  requirement  emphasizes  the  importance  of  validation  of  the 
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methods  and  the  need  for  statistically  based  sampling  strategies.  This 
emphasizes  the  need  for  methods  that  have  been  validated,  and  sampling 
strategies  that  are  likely  to  find  contamination  at  low  levels.  Probability- 
based  sampling  does  allow  conclusions,  at  specific  levels  of  confidence, 
about  testing  results. 

Using  a  probability-based  sampling  strategy,  together  with  validated 
methods  for  detecting  contamination,  would  provide  a  known  level  of 
confidence  with  which  to  interpret  any  negative  results.  This  would  allow 
agencies  to  be  more  definitive  in  determining  necessary  actions.  Figure  1 
shows  how  lack  of  validation  could  affect  individual  activities — including 
the  sampling  strategy — as  well  as  the  results  generated  by  the  overall 
process. 


Figure  1:  Lack  of  Validation  Can  Affect  Individual  Activities  and  the  Overall  Process 
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Source:  GAO  analysis  of  CDC,  EPA,  and  USPS  data. 
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The  lack  of  validated  methods  for  assessing  contamination  in  postal 
facilities  impeded  the  agencies  in  responding  to  the  incidents.  The 
significance  of  the  lack  of  validated  methods  was  exemplified  in  the  case 
of  the  one  postal  facility  where  negative  preliminary  results  were  obtained 
by  field-based  methods  of  analysis,  with  limitations  that  appear  not  to  have 
been  well  understood  by  some  agencies.  Negative  results  do  not 
necessarily  mean  a  facility  is  free  from  contamination.  As  we  reported, 
results  can  be  negative  if  (1)  samples  were  not  collected  from  places 
where  anthrax  was  present,  (2)  the  detection  limit  of  the  method  was 
greater  than  the  actual  contamination  level,  (3)  not  enough  samples  were 
recovered  from  the  sample  material,  (4)  analysis  of  the  sample  extract  did 
not  detect  spores,  or  (5)  anthrax  was  not  present  in  the  facility. 

In  addition,  while  the  2001  events  involved  anthrax,  many  other  biothreat 
agents  exist.  Differences  in  their  characteristics  mean  different  solutions. 
Accordingly,  efforts  to  develop  sampling  strategies  and  to  validate 
methods  should  address  requirements  specific  to  those  threat  agents  as 
well.  However,  since  addressing  other  agents  would  consume  resources 
and  time,  all  these  efforts  should  be  prioritized  in  a  long-term  strategy. 

The  several  agencies  that  dealt  with  the  anthrax  attacks  generally  worked 
well  together,  but  we  have  identified  areas  that  would  have  benefited  from 
one  agency’s  taking  the  lead  in  coordinating  the  response.  Given  the 
mission  of  DHS  and  its  responsibilities,  it  appears  that  DHS  is  now  well 
positioned  to  take  a  lead  role  in  promoting  and  coordinating  the  activities 
of  the  various  agencies  that  have  technical  expertise  related  to 
environmental  testing.  In  addition,  it  is  important  that  all  participating 
agencies  recognize  and  support  DHS  in  that  role  and  that  they  have  an 
effective  structure  for  participating  in  identifying  and  addressing  the 
appropriate  issues. 


Recommendations 
Executive  Action 


Given  the  lack  of  validated  methods  for  detecting  anthrax  contamination 
in  facilities,  we  recommended  in  our  2005  report  that  the  Secretary  of 
Homeland  Security  develop  a  coordinated  approach  to  (1)  improve  the 
overall  process  for  detecting  anthrax  and  (2)  increase  confidence  in 
negative  test  results  generated  by  that  process.  This  approach  would 
include  working  with  agencies  to  ensure  that  appropriate  validation 
studies  of  the  overall  process  of  sampling  activities,  including  the 
methods,  are  conducted.  Specifically,  we  recommended  that  the  Secretary 
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1.  take  a  lead  role  in  promoting  and  coordinating  the  activities  of  the 
various  agencies  that  have  the  technical  expertise  related  to 
environmental  testing; 

2.  ensure  that  a  definition  of  validation  is  developed  and  agreed  on; 

3.  guarantee  that  the  overall  process  of  sampling  activities,  including 
methods,  is  validated  so  that  performance  characteristics,  including 
limitations,  are  clearly  understood  and  results  can  be  correctly 
interpreted; 

4.  see  that  appropriate  investments  are  made  in  empirical  studies  to 
develop  probability-based  sampling  strategies  that  take  into  account 
the  complexities  of  indoor  environments; 

5.  ensure  that  appropriate,  prioritized  investments  are  made  for  all 
biothreat  agents;  and 

6.  make  sure  that  agency  policies,  procedures,  and  guidelines  reflect  the 
results  of  such  efforts. 

When  we  issued  our  report,  CDC,  DHS,  and  USPS  agreed  with  our 
conclusion — that  methods  for  detecting  anthrax  contamination  in  facilities 
were  not  validated — and  with  the  thrust  of  our  recommendations — calling 
for  a  coordinated,  systematic  effort  to  validate  the  methods  to  be  used  for 
such  testing.  But  they  (1)  disagreed  with  or  expressed  concern  about  our 
conclusions  or  the  recommendation  dealing  with  targeted  versus 
probability  sampling,  (2)  emphasized  that  validated  testing  methods  for 
anthrax  were  not  available  in  2001  and  that  federal  and  state  organizations 
did  the  best  they  could  under  the  circumstances,  and  (3)  identified  factors 
or  issues  that  need  to  be  considered  in  validating  testing  methods. 


Who  Is  Responsible 
for  Ensuring  and 
Guaranteeing  That 
Anthrax  Detection 
Methods  Will  Be 
Validated? 


After  we  issued  our  2005  report,  it  became  evident  that  there  was 
uncertainty  over  which  agency  would  take  the  lead  role  in  improving  the 
overall  process  for  detecting  anthrax  and  how  studies  were  to  be  funded. 
For  example,  DHS  stated  that  while  it  has  overall  responsibility  for 
coordinating  the  federal  response  during  future  biological  attacks,  EPA 
had  the  “primary  responsibility  for  establishing  the  strategies,  guidelines, 
and  plans  for  the  recovery  from  a  biological  attack”  and  HHS  had  the  lead 
role  for  any  related  public  health  response  and  guidelines.  DHS  also  stated 
that  it  coordinated  regularly  with  EPA’s  National  Homeland  Research 
Center  to  exchange  information  on  research  needs  and  to  discuss 
priorities  and  gaps  for  a  wide  range  of  security-related  research  areas. 
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DHS  stated  that  it  would  coordinate  with  EPA  to  ensure  that  appropriate 
investments  were  made  to  explore  improved  sampling.  However,  it  is 
unclear  to  us  how  DHS  would  ensure  that  appropriate  prioritized 
investments  are  made  for  all  biothreat  agents  and  how  such  priorities  and 
gaps  would  be  addressed. 

On  the  basis  of  these  uncertainties,  we  recommended  in  our  May  9,  2006, 
testimony  that  DHS’s  approach  to  validating  the  overall  process  should 
start  with  a  strategic  plan  that  includes  a  road  map  outlining  how 
individual  agencies  efforts  would  lead  to  the  validation  of  the  individual 
activities  as  well  as  the  overall  process,  noting  that  such  a  plan  would 
assist  DHS  in  monitoring  progress  and  measuring  agency  performance 
toward  improving  the  detection  of  anthrax  and  other  prioritized  threat 
agents. 

On  May  19,  2006,  DHS  officials  stated  that  while  they  concurred  with  the 
recommendations  from  our  report  and  accepted  the  overall  responsibility 
to  ensure  the  methods  will  be  validated,  they  stated  that  “there  are  legal 
limitations  in  DHS  authority  to  direct  the  activities  of  other  agencies.” 

They  said  that  while  they  take  a  lead  role  in  coordinating  the  meetings  and 
in  bringing  people  from  different  agencies  together,  they  cannot  guarantee 
that  the  overall  process  of  sampling  will  be  validated  because  different 
agencies  have  responsibility  for  different  aspects  of  validation,  and  DHS’s 
control  over  other  agencies  actions  and  budgets  is  ultimately  limited.  They 
stated  that  DHS  cannot  ensure  and  guarantee  that  validation  studies  would 
be  done,  since  this  is  a  shared  responsibility  among  different  agencies. 
Also,  since  validation  would  require  a  sustained  effort  over  a  long  period, 
DHS  noted  that  it  could  not  mandate  commitment  of  other  agencies’  funds, 
over  which  it  has  no  control. 

DHS  officials  told  us  in  July  2006  that  they  recognize  that  DHS  is  the 
principal  agency  responsible  for  coordinating  the  federal  response  and 
they  would  work  with  a  good  faith  effort  toward  developing  a  strategy  for 
validation  studies  and  a  road  map  by  the  end  of  calendar  year  2006 
outlining  how  individual  agencies’  efforts  would  lead  to  the  validation  of 
the  overall  sampling  process.  On  March  27,  2007,  DHS  told  us  that  it  had 
developed  a  working  draft  of  the  strategic  plan  and  the  road  map  by 
December  2006  but  it  could  not  share  these  with  us  because  they  were  not 
final. 

Until  responsibility  is  accepted  for  ensuring  that  sampling  activities  will  be 
validated,  the  fate  of  the  validation  process  will  remain  uncertain.  Without 
validation,  if  another  anthrax  attack  were  to  occur  tomorrow,  federal 
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civilian  agencies  would  not  be  able  to  conclude  with  any  given  level  of 
statistical  confidence,  in  cases  of  negative  results,  that  a  building  is  free  of 
contamination. 


Mr.  Chairman,  this  concludes  my  prepared  remarks.  I  would  be  happy  to 
respond  to  any  questions  that  you  or  other  members  of  the  subcommittee 
may  have  at  this  time. 
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